Creating IAM Roles for DataPipeline which is used for Movement of Data between DynamoDB and S3

DataPipeline

It is a service which that helps in sort, reformating, analyze, filtering, reporting data and deriving an outcome from it. It is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as On-Premises data sources, at specified intervals.

S3

Simple and popular AWS Service for storage. Replicates data by default across multiple facilities. It charges per usage. It is deeply integrated with AWS Services. Buckets are logical storage units. Objects are data added in the bucket. S3 has a storage class on object level which can save money by moving less frequently accessed objects to colder storage class.

DynamoDB

An Amazon AWS Web Services database which is fully managed, serverless, key-value NoSQL database, has high-performance applications at any scale. It has the following benefits such as built-in security, continuous backups, automated multi-region replication , in-memory caching, and data export tools

Demo

The following Roles must be created in the AWS account in advance inorder to make use of the AWS Data Pipeline

First IAM Role lets the pipeline take actions on behalf of the user

Log in to AWS and search IAM In search bar Click on role Click on create Role In the AWS Service select DataPipeline In the use cases click Data pipeline and click Permissions Click next until you reach the Review page, write a role name and click create role

2nd IAM Role this provisions AWS resources on behalf of the user. For import and export of DynamoDB data the Amazon EMR cluster resource and Amazon EC2 instances are associated with that cluster resource.

Login to AWS and search IAM In search bar Click on role Click on create Role In the AWS Service select Datapipeline In the use case select EC2 Role for Data Pipeline The attached Policies is already present Click next until you reach Review page, write a role name and click create role